p-Automata: Acceptors for Markov Chains

We present p-automata, which accept an entire Markovchain as input. Acceptance is determined by solving a se-quence of stochastic weak and weak games. The set oflanguages of Markov chains obtained in this way is closedunder Boolean operations. Language emptiness and con-tainment are equi-solvable, and languages themselves areclosed under bisimulation. A Markov chain (respectively,PCTL formula) determines a p-automaton whose languageis the bisimulation equivalence class of that Markov chain(respectively, the set of models of that formula). We definea simulation game between p-automata, decidable in EX-PTIME. Simulation under-approximates language contain-ment, whose decidability status is presently unknown.